Sample-wise dynamic precision quantization for neural network acceleration

Quantization is a well-known method for deep neural networks (DNNs) compression and acceleration. In this work, we propose the Sample-Wise Dynamic Precision (SWDP) quantization scheme, which can switch bit-width of weights activations in model according to task difficulty input samples at runtime. Using low-precision easy images brings advantages terms computational energy efficiency. We also an adaptive hardware design efficient implementation our SWDP networks. The experimental results on various datasets demonstrate that achieves average 3.3× speedup 3.0× saving over bit-level dynamically composable architecture BitFusion.